Papers by Ashish V. Thapliyal
End-to-end Dense Video Captioning as Sequence Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for dense video captioning use a two-stage generative process . but, more complex tasks are not able to fully utilize this powerful paradigm . |
| Approach: | They propose to model two subtasks of dense video captioning as one sequence generation task and predict the events and the corresponding descriptions. |
| Outcome: | Experiments on YouCook2 and ViTT show that the proposed model can be used on any video platform. |
Denoising Large-Scale Image Captioning from Alt-text Data Using Content Selection Models (2022.coling-1)
Copied to clipboard
| Challenge: | Recent approaches to training large-scale image captioning (IC) models often fall short in terms of performance in leveraging noisy datasets in favor of clean annotations. |
| Approach: | They propose a technique that breaks down the task into two smaller, more controllable tasks - skeleton prediction and skelet-based caption generation. |
| Outcome: | The proposed method can generate better and denoised captions when using noisy datasets. |
Quality Estimation for Image Captions Based on Large-scale Human Evaluations (2021.naacl-main)
Copied to clipboard
| Challenge: | a problem with automatic image captioning is that it produces low quality captions when used in the wild. |
| Approach: | They propose to model caption quality from a human perspective and *without* access to ground-truth references. |
| Outcome: | The proposed model can detect and filter out low-quality captions on previously unseen images. |
Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Dataset (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on multilingual image captioning have been hampered by a lack of high-quality evaluation datasets. |
| Approach: | They present a dataset of 3600 images annotated with human-generated captions in 36 languages. |
| Outcome: | The proposed dataset shows that it is feasible to build multilingual image captioning models trained on machine-translated data. |
Cross-modal Language Generation using Pivot Stabilization for Web-scale Language Coverage (2020.acl-main)
Copied to clipboard
| Challenge: | Existing datasets have extensive labeled data for En-glish, but labeles are extremely scarce in other languages. |
| Approach: | They propose a method that leverages existing annotations with machine translation capabilities to create cross-modal language generation systems at web-scale. |
| Outcome: | The proposed model outperforms other candidates in evaluations performed over 5 target languages. |